SherbyNet - /page/wikibook/entry.mu

<!DOCTYPE html>

<title>Model collapse</title>

</head>

Model collapse

</h1>

</header>

Model collapse<a href="#cite_note-9">[note 1]</a> is a phenomenon where <a href="Machine_learning" title="Machine learning">machine learning</a> models gradually degrade due to errors coming from uncurated training on the outputs of another model, such as prior versions of itself.<a href="#cite_note-Shumailov-2024-10">[9]</a><a href="#cite_note-11">[10]</a><a href="#cite_note-12">[11]</a><a href="#cite_note-13">[12]</a> Such outputs are known as <a href="Synthetic_data" title="Synthetic data">synthetic data</a>. It is a possible mechanism for <a href="Mode_collapse" title="Mode collapse">mode collapse</a>.

Shumailov et al.<a href="#cite_note-Shumailov-2024-10">[9]</a> coined the term and described two specific stages to the degradation: early model collapse and late model collapse:

<ul><li>In early model collapse, the model begins losing information about the tails of the <a href="Probability_distribution" title="Probability distribution">distribution</a> – mostly affecting minority data. Later work highlighted that early model collapse is hard to notice, since overall performance may appear to improve, while the model loses performance on minority data.<a href="#cite_note-14">[13]</a></li>

<li>In late model collapse, the model loses a significant proportion of its performance, confusing concepts and losing most of its <a href="Variance" title="Variance">variance</a>.</li></ul>

<div class="mw-heading mw-heading2"><h2 id="Mechanism">Mechanism</h2></div>

Using synthetic data as training data can lead to issues with the quality and reliability of the trained model.<a href="#cite_note-Alemohammad-2023-15">[14]</a><a href="#cite_note-16">[15]</a> Model collapse occurs for three main reasons:

<ol><li>functional approximation errors</li>

<li>sampling errors</li>

<li>learning errors<a href="#cite_note-Shumailov-2024-10">[9]</a></li></ol>

Importantly, it happens in even the simplest of models, where not all of the error sources are present. In more complex models the errors often compound, leading to faster collapse.

<div class="mw-heading mw-heading2"><h2 id="Disagreement_over_real-world_impact">Disagreement over real-world impact</h2></div>

Some researchers and commentators on model collapse warn that the phenomenon could fundamentally threaten future generative AI development: As AI-generated data is shared on the Internet, it will inevitably end up in future training datasets, which are often crawled from the Internet. If training on "<a href="Slop_(artificial_intelligence)" class="mw-redirect" title="Slop (artificial intelligence)">slop</a>" (large quantities of unlabeled synthetic data) inevitably leads to model collapse, this could therefore pose a difficult problem.<a href="#cite_note-17">[16]</a>

However, recently, other researchers have disagreed with this argument, showing that if synthetic data accumulates alongside human-generated data, model collapse is avoided.<a href="#cite_note-18">[17]</a> The researchers argue that data accumulating over time is a more realistic description of reality than deleting all existing data every year, and that the real-world impact of model collapse may not be as catastrophic as feared.<a href="#cite_note-19">[18]</a>

An alternative branch of the literature investigates the use of machine learning detectors and watermarking to identify model generated data and filter it out.<a href="#cite_note-20">[19]</a><a href="#cite_note-21">[20]</a>

<div class="mw-heading mw-heading2"><h2 id="Mathematical_models_of_the_phenomenon">Mathematical models of the phenomenon</h2></div>

<div class="mw-heading mw-heading3"><h3 id="1D_Gaussian_model">1D Gaussian model</h3></div>

In 2024,<a href="#cite_note-Shumailov-2024-10">[9]</a> a first attempt has been made at illustrating collapse for the simplest possible model — a single dimensional <a href="Normal_distribution" title="Normal distribution">normal distribution</a> fit using <a href="Unbiased_estimator" class="mw-redirect" title="Unbiased estimator">unbiased estimators</a> of mean and variance, computed on samples from the previous generation.

To make this more precise, we say that original data follows a normal distribution <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X^{0}\sim {\mathcal {N}}(\mu ,\sigma ^{2})}">

<msup>

</mrow>

</msup>

</mrow>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X^{0}\sim {\mathcal {N}}(\mu ,\sigma ^{2})}</annotation>

</semantics>

</math><img src="./8decbb31e79ce0610ed35f96d7fa3a1fedc9ec42.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:15.054ex; height:3.176ex;" alt="{\displaystyle X^{0}\sim {\mathcal {N}}(\mu ,\sigma ^{2})}" loading="lazy">, and we possess <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle M_{0}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle M_{0}}</annotation>

</semantics>

</math><img src="./909a62f3ff41169372143733d3767afe0ad3b14d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.308ex; height:2.509ex;" alt="{\displaystyle M_{0}}" loading="lazy"> samples <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X_{j}^{0}}">

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X_{j}^{0}}</annotation>

</semantics>

</math><img src="./7cc9ed9537ee4b1d0f3ab9e8d9433aa7cdb72910.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.338ex; width:3.051ex; height:3.509ex;" alt="{\displaystyle X_{j}^{0}}" loading="lazy"> for <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle j\in {\{\,1,\dots ,M_{0}\,{}\}}}">

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle j\in {\{\,1,\dots ,M_{0}\,{}\}}}</annotation>

</semantics>

</math><img src="./d3b169356db5689ca5ac474994e49d7020b4c597.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; margin-left: -0.027ex; width:16.574ex; height:2.843ex;" alt="{\displaystyle j\in {\{\,1,\dots ,M_{0}\,{}\}}}" loading="lazy">. Denoting a general sample <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X_{j}^{i}}">

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X_{j}^{i}}</annotation>

</semantics>

</math><img src="./2ecdacb8a36a9ab50443af87707dbc2df361d796.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.338ex; width:2.834ex; height:3.509ex;" alt="{\displaystyle X_{j}^{i}}" loading="lazy"> as sample <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle j\in {\{\,1,\dots ,M_{i}\,{}\}}}">

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle j\in {\{\,1,\dots ,M_{i}\,{}\}}}</annotation>

</semantics>

</math><img src="./8fd8043e478b8430a983e51eea8150fb3001d37e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; margin-left: -0.027ex; width:16.319ex; height:2.843ex;" alt="{\displaystyle j\in {\{\,1,\dots ,M_{i}\,{}\}}}" loading="lazy"> at generation <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle i}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle i}</annotation>

</semantics>

</math><img src="./add78d8608ad86e54951b8c8bd6c8d8416533d20.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:0.802ex; height:2.176ex;" alt="{\displaystyle i}" loading="lazy">, then the next generation model is estimated using the sample mean and variance:

<msub>

</mrow>

</msub>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mrow>

</munder>

</mrow>

</mrow>

</msubsup>

</mrow>

</mrow>

</msubsup>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mrow>

</munder>

</mrow>

</mrow>

</msubsup>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mu _{i+1}={\frac {1}{M_{i}}}\sum _{j}X_{j}^{i};\quad \sigma _{i+1}^{2}={\frac {1}{M_{i}-1}}\sum _{j}(X_{j}^{i}-\mu _{i+1})^{2}.}</annotation>

</semantics>

</math><img src="./a9f9bf06b4c9b1373dd716737bf579ad8960cefb.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.338ex; width:54.057ex; height:6.676ex;" alt="{\displaystyle \mu _{i+1}={\frac {1}{M_{i}}}\sum _{j}X_{j}^{i};\quad \sigma _{i+1}^{2}={\frac {1}{M_{i}-1}}\sum _{j}(X_{j}^{i}-\mu _{i+1})^{2}.}" loading="lazy">

Leading to a conditionally normal next generation model <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X_{j}^{i+1}|\mu _{i+1},\;\sigma _{i+1}\sim {\mathcal {N}}(\mu _{i+1},\sigma _{i+1}^{2})}">

</mrow>

</mrow>

</msubsup>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X_{j}^{i+1}|\mu _{i+1},\;\sigma _{i+1}\sim {\mathcal {N}}(\mu _{i+1},\sigma _{i+1}^{2})}</annotation>

</semantics>

</math><img src="./c3e6170255c738631bccbb838a078bef7690b978.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.338ex; width:32.499ex; height:3.676ex;" alt="{\displaystyle X_{j}^{i+1}|\mu _{i+1},\;\sigma _{i+1}\sim {\mathcal {N}}(\mu _{i+1},\sigma _{i+1}^{2})}" loading="lazy">. In theory, this is enough to calculate the full distribution of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X_{j}^{i}}">

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X_{j}^{i}}</annotation>

</semantics>

</math><img src="./2ecdacb8a36a9ab50443af87707dbc2df361d796.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.338ex; width:2.834ex; height:3.509ex;" alt="{\displaystyle X_{j}^{i}}" loading="lazy">. However, even after the first generation, the full distribution is no longer normal: It follows a <a href="Variance-gamma_distribution" title="Variance-gamma distribution">variance-gamma distribution</a>.

To continue the analysis, instead of writing the probability density function at each generation, it is possible to explicitly construct them in terms of independent random variables using <a href="Cochran's_theorem" title="Cochran's theorem">Cochran's theorem</a>. To be precise, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mu _{1}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mu _{1}}</annotation>

</semantics>

</math><img src="./d6899621035d3359b9c1c064739b54c7004e220d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:2.456ex; height:2.176ex;" alt="{\displaystyle \mu _{1}}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \sigma _{1}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \sigma _{1}}</annotation>

</semantics>

</math><img src="./7fa0e56273a1cb32709b442e2421e9f947522b84.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.382ex; height:2.009ex;" alt="{\displaystyle \sigma _{1}}" loading="lazy">are independent, with <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mu _{1}\sim {\mathcal {N}}\left(\mu ,{\frac {\sigma ^{2}}{M_{0}}}\right)}">

<msub>

</mrow>

</msub>

</mrow>

<mrow>

<mrow>

<mfrac>

<msup>

</mrow>

</msup>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mu _{1}\sim {\mathcal {N}}\left(\mu ,{\frac {\sigma ^{2}}{M_{0}}}\right)}</annotation>

</semantics>

</math><img src="./85ca02148257c6f26f88fde68b8d8679b3818bf0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.505ex; width:18.218ex; height:6.343ex;" alt="{\displaystyle \mu _{1}\sim {\mathcal {N}}\left(\mu ,{\frac {\sigma ^{2}}{M_{0}}}\right)}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (M_{0}-1)\,\sigma _{1}^{2}\sim \sigma ^{2}\,\Gamma \left({\frac {M_{0}-1}{2}},{\frac {1}{2}}\right)}">

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</msubsup>

<msup>

</mrow>

</msup>

<mrow>

<mrow>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

<mfrac>

</mfrac>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (M_{0}-1)\,\sigma _{1}^{2}\sim \sigma ^{2}\,\Gamma \left({\frac {M_{0}-1}{2}},{\frac {1}{2}}\right)}</annotation>

</semantics>

</math><img src="./3eeeb20529412ab6a1caee352f5412b2ad0ef6b3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.505ex; width:34.203ex; height:6.176ex;" alt="{\displaystyle (M_{0}-1)\,\sigma _{1}^{2}\sim \sigma ^{2}\,\Gamma \left({\frac {M_{0}-1}{2}},{\frac {1}{2}}\right)}" loading="lazy">, following a <a href="Gamma_distribution" title="Gamma distribution">Gamma distribution</a>. Denoting with <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Z}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Z}</annotation>

</semantics>

</math><img src="./1cc6b75e09a8aa3f04d8584b11db534f88fb56bd.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.68ex; height:2.176ex;" alt="{\displaystyle Z}" loading="lazy"> Gaussian random variables distributed according to <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {N}}(0,1)}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mathcal {N}}(0,1)}</annotation>

</semantics>

</math><img src="./a3eeb356405c0b33b680b5caa425ada4e9f53e8b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; margin-left: -0.062ex; width:7.505ex; height:3.009ex;" alt="{\displaystyle {\mathcal {N}}(0,1)}" loading="lazy"> and with <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle S^{i}}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle S^{i}}</annotation>

</semantics>

</math><img src="./5ca697409cb6c118337963c06933271a6f33c031.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.321ex; height:2.676ex;" alt="{\displaystyle S^{i}}" loading="lazy"> random variables distributed with <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\frac {1}{M_{i-1}-1}}\Gamma \left({\frac {M_{i-1}-1}{2}},{\frac {1}{2}}\right)}">

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

<mrow>

<mrow>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

<mfrac>

</mfrac>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\frac {1}{M_{i-1}-1}}\Gamma \left({\frac {M_{i-1}-1}{2}},{\frac {1}{2}}\right)}</annotation>

</semantics>

</math><img src="./7234837f857c04a12aeb040fae1091a10b0af216.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.505ex; width:28.28ex; height:6.176ex;" alt="{\displaystyle {\frac {1}{M_{i-1}-1}}\Gamma \left({\frac {M_{i-1}-1}{2}},{\frac {1}{2}}\right)}" loading="lazy">, it turns out to be possible to write samples at each generation as

</mrow>

</mrow>

</msubsup>

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle X_{j}^{0}=\mu +\sigma Z_{j}^{0},}</annotation>

</semantics>

</math><img src="./8b0f0f3d94157cba67e085b5c28ef553b1a39916.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.338ex; width:15.131ex; height:3.509ex;" alt="{\textstyle X_{j}^{0}=\mu +\sigma Z_{j}^{0},}" loading="lazy">

</mrow>

</mrow>

</msubsup>

<mfrac>

<msqrt>

<msub>

</mrow>

</msub>

</msqrt>

</mfrac>

</mrow>

<msup>

</mrow>

</msup>

<msqrt>

<msup>

</mrow>

</msup>

</msqrt>

</mrow>

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle X_{j}^{1}=\mu +{\frac {\sigma }{\sqrt {M_{0}}}}Z^{1}+\sigma {\sqrt {S^{1}}}Z_{j}^{1},}</annotation>

</semantics>

</math><img src="./9b1918e16e340121a4fe36627b694531a33031cc.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.171ex; width:30.15ex; height:4.676ex;" alt="{\textstyle X_{j}^{1}=\mu +{\frac {\sigma }{\sqrt {M_{0}}}}Z^{1}+\sigma {\sqrt {S^{1}}}Z_{j}^{1},}" loading="lazy">

and more generally

</mrow>

</mrow>

</msubsup>

<mfrac>

<msqrt>

<msub>

</mrow>

</msub>

</msqrt>

</mfrac>

</mrow>

<msup>

</mrow>

</msup>

<mfrac>

<msqrt>

<msub>

</mrow>

</msub>

</msqrt>

</mfrac>

</mrow>

<msqrt>

<msup>

</mrow>

</msup>

</msqrt>

</mrow>

<msup>

</mrow>

</msup>

<mfrac>

<msqrt>

<msub>

</mrow>

</msub>

</msqrt>

</mfrac>

</mrow>

<msqrt>

<msup>

</mrow>

</msup>

<msup>

</mrow>

</msup>

</msqrt>

</mrow>

<msup>

</mrow>

</msup>

<msqrt>

<msup>

</mrow>

</msup>

<msup>

</mrow>

</msup>

</msqrt>

</mrow>

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X_{j}^{n}=\mu +{\frac {\sigma }{\sqrt {M_{0}}}}Z^{1}+{\frac {\sigma }{\sqrt {M_{1}}}}{\sqrt {S^{1}}}Z^{2}+\dots +{\frac {\sigma }{\sqrt {M_{n-1}}}}{\sqrt {S^{1}\times \dots \times S^{n-1}}}Z^{n}+\sigma {\sqrt {S^{1}\times \dots \times S^{n}}}Z_{j}^{n}.}</annotation>

</semantics>

</math>

Note, that these are not joint distributions, as <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Z^{n}}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Z^{n}}</annotation>

</semantics>

</math><img src="./d4183afbbcfca3b02b3417fc664ea5e5236ed22e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.927ex; height:2.343ex;" alt="{\displaystyle Z^{n}}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle S^{n}}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle S^{n}}</annotation>

</semantics>

</math><img src="./ee006452a59bf1eb29983b4412348b66517a2d23.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.74ex; height:2.343ex;" alt="{\displaystyle S^{n}}" loading="lazy"> depend directly on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Z_{j}^{n-1}}">

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Z_{j}^{n-1}}</annotation>

</semantics>

</math><img src="./0cbf604a3b2d72faa7d79ebf0b6e59c7b3c6f1ea.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.338ex; width:5.027ex; height:3.676ex;" alt="{\displaystyle Z_{j}^{n-1}}" loading="lazy">, but when considering <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X_{j}^{n}}">

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X_{j}^{n}}</annotation>

</semantics>

</math><img src="./bf12f407d9773a094c48fbe567fcc7fa582e605b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.338ex; width:3.215ex; height:3.176ex;" alt="{\displaystyle X_{j}^{n}}" loading="lazy"> on its own the formula above provides all the information about the full distribution.

To analyse the model collapse, we can first calculate variance and mean of samples at generation <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy">. This would tell us what kind of distributions we expect to arrive at after <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy"> generations. It is possible to find its exact value in closed form, but the mean and variance of the square root of gamma distribution are expressed in terms of gamma functions, making the result quite clunky. Following,<a href="#cite_note-Shumailov-2024-10">[9]</a> it is possible to expand all results to second order in each of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle 1/M_{i}}">

</mrow>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle 1/M_{i}}</annotation>

</semantics>

</math><img src="./e0cef277fd1b2fb27bc87eb60cb725fd1888a83e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:5.379ex; height:2.843ex;" alt="{\displaystyle 1/M_{i}}" loading="lazy">, assuming each sample size to be large. It is then possible to show that

<mfrac>

<msup>

</mrow>

</msup>

</mfrac>

</mrow>

</mrow>

</mrow>

</msubsup>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mrow>

<mrow>

</mrow>

</mrow>

</msubsup>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\frac {1}{\sigma ^{2}}}\operatorname {Var} (X_{j}^{n})={\frac {1}{M_{0}}}+{\frac {1}{M_{1}}}+\dots +{\frac {1}{M_{n-1}}}+1+{\mathcal {O}}\left(M_{i}^{-2}\right).}</annotation>

</semantics>

</math><img src="./d8e07c1e501bed5e7e5fb438b15ec27ec9f63306.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.338ex; width:58.566ex; height:5.676ex;" alt="{\displaystyle {\frac {1}{\sigma ^{2}}}\operatorname {Var} (X_{j}^{n})={\frac {1}{M_{0}}}+{\frac {1}{M_{1}}}+\dots +{\frac {1}{M_{n-1}}}+1+{\mathcal {O}}\left(M_{i}^{-2}\right).}" loading="lazy">

And if all sample sizes <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle M_{i}=M}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle M_{i}=M}</annotation>

</semantics>

</math><img src="./b2b261ba55c0c48baf212471be1e04262e5eb75d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:8.594ex; height:2.509ex;" alt="{\displaystyle M_{i}=M}" loading="lazy"> are constant, this diverges linearly as <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n\to \infty }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n\to \infty }</annotation>

</semantics>

</math><img src="./a0d55d9b32f6fa8fab6a84ea444a6b5a24bb45e1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:7.333ex; height:1.843ex;" alt="{\displaystyle n\to \infty }" loading="lazy">:

</mrow>

</mrow>

</msubsup>

<msup>

</mrow>

</msup>

<mrow>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mrow>

</mrow>

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \operatorname {Var} (X_{j}^{n})=\sigma ^{2}\left(1+{\frac {n}{M}}\right);\quad \mathbb {E} (X_{j}^{n})=\mu .}</annotation>

</semantics>

</math><img src="./7f30e5e68199a752a4d98fba6f65009eee9dd28c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; width:40.234ex; height:4.843ex;" alt="{\displaystyle \operatorname {Var} (X_{j}^{n})=\sigma ^{2}\left(1+{\frac {n}{M}}\right);\quad \mathbb {E} (X_{j}^{n})=\mu .}" loading="lazy">

This is the same scaling as for a single dimensional <a href="Random_walk" title="Random walk">Gaussian random walk</a>. However, divergence of the variance of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X_{j}^{n}}">

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X_{j}^{n}}</annotation>

</semantics>

</math><img src="./bf12f407d9773a094c48fbe567fcc7fa582e605b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.338ex; width:3.215ex; height:3.176ex;" alt="{\displaystyle X_{j}^{n}}" loading="lazy"> does not directly provide any information about the corresponding estimates of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mu _{n+1}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mu _{n+1}}</annotation>

</semantics>

</math><img src="./73729c7d4f8e8ab14bc70136aeca9330ab8f857f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.721ex; height:2.176ex;" alt="{\displaystyle \mu _{n+1}}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \sigma _{n+1}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \sigma _{n+1}}</annotation>

</semantics>

</math><img src="./0dfaf1621bc4a3a3432673bad1afdc122046d5b5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:4.646ex; height:2.009ex;" alt="{\displaystyle \sigma _{n+1}}" loading="lazy">, particularly how different they are from the original <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mu }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mu }</annotation>

</semantics>

</math><img src="./9fd47b2a39f7a7856952afec1f1db72c67af6161.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:1.402ex; height:2.176ex;" alt="{\displaystyle \mu }" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \sigma }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \sigma }</annotation>

</semantics>

</math><img src="./59f59b7c3e6fdb1d0365a494b81fb9a696138c36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.33ex; height:1.676ex;" alt="{\displaystyle \sigma }" loading="lazy">. It turns out to be possible to calculate the distance between the true distribution and the approximated distribution at step <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n+1}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n+1}</annotation>

</semantics>

</math><img src="./2a135e65a42f2d73cccbfc4569523996ca0036f1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.505ex; width:5.398ex; height:2.343ex;" alt="{\displaystyle n+1}" loading="lazy">, using the <a href="Wasserstein_distance" class="mw-redirect" title="Wasserstein distance">Wasserstein-2 distance</a> (which is also sometimes referred to as <a href="Empirical_risk_minimization" title="Empirical risk minimization">risk</a>):

</mrow>

<mrow>

<mrow>

</mrow>

</mrow>

</mrow>

</msubsup>

<mrow>

<mrow>

</mrow>

<msup>

</mrow>

</msup>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</msubsup>

</mrow>

</mrow>

</mrow>

<mfrac>

</mfrac>

</mrow>

<msup>

</mrow>

</msup>

<mrow>

<mrow>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mrow>

</mrow>

<mrow>

</mrow>

</mrow>

</msubsup>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathbb {E} \left[\mathbb {W} _{2}^{2}\left({\mathcal {N}}(\mu ,\sigma ^{2}),{\mathcal {N}}(\mu _{n+1},\sigma _{n+1}^{2})\right)\right]={\frac {3}{2}}\sigma ^{2}\left({\frac {1}{M_{0}}}+{\frac {1}{M_{1}}}+\dots +{\frac {1}{M_{n}}}\right)+{\mathcal {O}}\left(M_{i}^{-2}\right),}</annotation>

</semantics>

</math><img src="./de541cb21bb75f461b79be645a7a00b9fa3c2a0b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.505ex; width:82.4ex; height:6.176ex;" alt="{\displaystyle \mathbb {E} \left[\mathbb {W} _{2}^{2}\left({\mathcal {N}}(\mu ,\sigma ^{2}),{\mathcal {N}}(\mu _{n+1},\sigma _{n+1}^{2})\right)\right]={\frac {3}{2}}\sigma ^{2}\left({\frac {1}{M_{0}}}+{\frac {1}{M_{1}}}+\dots +{\frac {1}{M_{n}}}\right)+{\mathcal {O}}\left(M_{i}^{-2}\right),}" loading="lazy">

<mrow>

<mrow>

</mrow>

</mrow>

</mrow>

</msubsup>

<mrow>

<mrow>

</mrow>

<msup>

</mrow>

</msup>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</msubsup>

</mrow>

</mrow>

</mrow>

<mfrac>

</mfrac>

</mrow>

<msup>

</mrow>

</msup>

<mrow>

<mrow>

<mfrac>

</mrow>

</mrow>

</msubsup>

</mfrac>

</mrow>

<mfrac>

</mrow>

</mrow>

</msubsup>

</mfrac>

</mrow>

<mfrac>

</mrow>

</mrow>

</msubsup>

</mfrac>

</mrow>

</mrow>

</munder>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mrow>

</mrow>

<mrow>

</mrow>

</mrow>

</msubsup>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \operatorname {Var} \left[\mathbb {W} _{2}^{2}\left({\mathcal {N}}(\mu ,\sigma ^{2}),{\mathcal {N}}(\mu _{n+1},\sigma _{n+1}^{2})\right)\right]={\frac {1}{2}}\sigma ^{4}\left({\frac {3}{M_{0}^{2}}}+{\frac {3}{M_{1}^{2}}}+\dots +{\frac {3}{M_{n}^{2}}}+\sum _{i\neq j}{\frac {4}{M_{i}M_{j}}}\right)+{\mathcal {O}}\left(M_{i}^{-3}\right).}</annotation>

</semantics>

</math><img src="./fd2a0f42388cce1f2b28facb031c714875abf00d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.505ex; width:98.746ex; height:7.843ex;" alt="{\displaystyle \operatorname {Var} \left[\mathbb {W} _{2}^{2}\left({\mathcal {N}}(\mu ,\sigma ^{2}),{\mathcal {N}}(\mu _{n+1},\sigma _{n+1}^{2})\right)\right]={\frac {1}{2}}\sigma ^{4}\left({\frac {3}{M_{0}^{2}}}+{\frac {3}{M_{1}^{2}}}+\dots +{\frac {3}{M_{n}^{2}}}+\sum _{i\neq j}{\frac {4}{M_{i}M_{j}}}\right)+{\mathcal {O}}\left(M_{i}^{-3}\right).}" loading="lazy">

This directly shows why model collapse occurs in this simple model. Due to errors from re-sampling the approximated distribution, each generation ends up corresponding to a new step in a random walk of model parameters. For a constant sample size at each generation, the average distance from the starting point diverges, and in order for the end distribution approximation to be accurate, or for the distance to be finite, the sampling rate <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle M_{i}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle M_{i}}</annotation>

</semantics>

</math><img src="./eda8fd06f1cd5de22ed07385a0f8aa19773b2de9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.054ex; height:2.509ex;" alt="{\displaystyle M_{i}}" loading="lazy"> needs to increase superlinearly, i.e. one needs to collect increasingly more samples over time, perhaps quadratically. However, even in that case the expected distance after <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy"> steps remains non-zero and the only case in which it does in fact end up being zero is when sampling is infinite at each step. Overall, this only shows us how far on average one ends up from the original distribution, but the process can only "terminate", if the estimated variance at a certain generation becomes small enough, effectively turning the distribution into a delta function. This is shown to occur for a general gaussian model<a href="#cite_note-Alemohammad-2023-15">[14]</a> in the subsection below. Empirical investigation has confirmed this theoretical analysis.<a href="#cite_note-22">[21]</a>

<div class="mw-heading mw-heading3"><h3 id="N-D_Gaussian_model">N-D Gaussian model</h3></div>

Furthermore, in the case of multidimensional model with fully synthetic data, exact collapse can be shown.<a href="#cite_note-Alemohammad-2023-15">[14]</a><a href="#cite_note-Shumailov-2024-10">[9]</a>

<div class="mw-heading mw-heading3"><h3 id="Linear_regression">Linear regression</h3></div>

In the case of a <a href="Linear_regression" title="Linear regression">linear regression</a> model,<a href="#cite_note-23">[22]</a><a href="#cite_note-24">[23]</a> <a href="Power_law" title="Power law">scaling laws</a> and bounds on learning can be obtained.

<div class="mw-heading mw-heading3"><h3 id="Statistical_language_model">Statistical language model</h3></div>

In the case of a linear <a href="Softmax" class="mw-redirect" title="Softmax">softmax</a> classifier for next token prediction,<a href="#cite_note-25">[24]</a> exact bounds on learning with even a partially synthetic dataset can be obtained.

<div class="mw-heading mw-heading2"><h2 id="Impact_on_large_language_models">Impact on large language models</h2></div>

In the context of <a href="Large_language_model" title="Large language model">large language models</a>, research found that training LLMs on predecessor-generated text — language models are trained on the synthetic data produced by previous models — causes a consistent decrease in the lexical, syntactic, and semantic diversity of the model outputs through successive iterations, notably remarkable for tasks demanding high levels of creativity.<a href="#cite_note-26">[25]</a>

<ul><li><a href="Generation_loss" title="Generation loss">Generation loss</a></li>

<li><a href="Generative_artificial_intelligence" title="Generative artificial intelligence">Generative artificial intelligence</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="Notes">Notes</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist">

<li id="cite_note-9"><a href="#cite_ref-9">^</a> Also known by other names, such as "AI <a href="Inbreeding" title="Inbreeding">inbreeding</a>",<a href="#cite_note-1">[1]</a><a href="#cite_note-2">[2]</a> "AI <a href="Cannibalism" title="Cannibalism">cannibalism</a>",<a href="#cite_note-3">[3]</a><a href="#cite_note-4">[4]</a> "<a href="House_of_Habsburg#Habsburg_inbreeding_and_extinction_of_the_male_lines" title="House of Habsburg">Habsburg</a> AI",<a href="#cite_note-5">[5]</a> and "model <a href="Autophagy" title="Autophagy">autophagy</a> disorder", abbreviated "MAD"<a href="#cite_note-6">[6]</a><a href="#cite_note-7">[7]</a><a href="#cite_note-8">[8]</a>

</li>

</ol></div></div>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://venturebeat.com/ai/generative-inbreeding-and-its-risk-to-human-culture/">"'Generative inbreeding' and its risk to human culture"</a>. 26 August 2023.</cite>

</li>

<li id="cite_note-2"><a href="#cite_ref-2">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.axios.com/2023/08/28/ai-content-flood-model-collapse">"AI could choke on its own exhaust as it fills the web"</a>. 28 August 2023.</cite>

</li>

<li id="cite_note-3"><a href="#cite_ref-3">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://ctlj.colorado.edu/?p=1280">"AI Cannibalism and the Law – Colorado Technology Law Journal"</a>.</cite>

</li>

<li id="cite_note-4"><a href="#cite_ref-4">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.cogitotech.com/blog/the-curious-case-of-ai-cannibalism-and-possible-solutions/">"The Curious Case of AI Cannibalism & Possible Solutions"</a>. 26 July 2023.</cite>,

</li>

<li id="cite_note-5"><a href="#cite_ref-5">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.france24.com/en/live-news/20240805-inbred-gibberish-or-just-mad-warnings-rise-about-ai-models">"Inbred, gibberish or just MAD? Warnings rise about AI models"</a>. France 24. 2024-08-05. Retrieved 2024-12-31.</cite>

</li>

<li id="cite_note-6"><a href="#cite_ref-6">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://livescu.ucla.edu/model-autophagy-disorder/">"Model Autophagy Disorder – the Livescu Initiative on Neuro, Narrative and AI"</a>.</cite>

</li>

<li id="cite_note-7"><a href="#cite_ref-7">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.tomshardware.com/news/generative-ai-goes-mad-when-trained-on-artificial-data-over-five-times">"Generative AI Goes 'MAD' when Trained on AI-Created Data over Five Times"</a>. 12 July 2023.</cite>

</li>

<li id="cite_note-8"><a href="#cite_ref-8">^</a> <cite id="CITEREFAlemohammadCasco-RodriguezLuziAhmed_Imtiaz_Humayun2023" class="citation arxiv cs1">Alemohammad, Sina; Casco-Rodriguez, Josue; Luzi, Lorenzo; Ahmed Imtiaz Humayun; Babaei, Hossein; LeJeune, Daniel; Siahkoohi, Ali; Baraniuk, Richard G. (2023). "Self-Consuming Generative Models Go MAD". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2307.01850">2307.01850</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.LG">cs.LG</a>].</cite>

</li>

<li id="cite_note-Shumailov-2024-10">^ <a href="#cite_ref-Shumailov-2024_10-0">a</a> <a href="#cite_ref-Shumailov-2024_10-1">b</a> <a href="#cite_ref-Shumailov-2024_10-2">c</a> <a href="#cite_ref-Shumailov-2024_10-3">d</a> <a href="#cite_ref-Shumailov-2024_10-4">e</a> <a href="#cite_ref-Shumailov-2024_10-5">f</a> <cite id="CITEREFShumailovShumaylovZhaoPapernot2024" class="citation journal cs1">Shumailov, Ilia; Shumaylov, Zakhar; Zhao, Yiren; Papernot, Nicolas; Anderson, Ross; Gal, Yarin (July 2024). <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11269175">"AI models collapse when trained on recursively generated data"</a>. Nature. 631 (8022): 755–759. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2024Natur.631..755S">2024Natur.631..755S</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1038%2Fs41586-024-07566-y">10.1038/s41586-024-07566-y</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1476-4687">1476-4687</a>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11269175">11269175</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/39048682">39048682</a>.</cite>

</li>

<li id="cite_note-11"><a href="#cite_ref-11">^</a> <cite id="CITEREFShumailovShumaylovZhaoGal2023" class="citation arxiv cs1">Shumailov, Ilia; Shumaylov, Zakhar; Zhao, Yiren; Gal, Yarin; Papernot, Nicolas; Anderson, Ross (2023-05-31). "The Curse of Recursion: Training on Generated Data Makes Models Forget". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2305.17493">2305.17493</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.LG">cs.LG</a>].</cite>

</li>

<li id="cite_note-12"><a href="#cite_ref-12">^</a> <cite id="CITEREFOzsevim2023" class="citation news cs1">Ozsevim, Ilkhan (2023-06-20). <a rel="nofollow" class="external text" href="https://aimagazine.com/articles/research-finds-chatgpt-headed-for-model-collapse">"Research finds ChatGPT & Bard headed for 'Model Collapse'"</a>. Retrieved 2024-03-06.</cite>

</li>

<li id="cite_note-13"><a href="#cite_ref-13">^</a> <cite id="CITEREFMok" class="citation web cs1">Mok, Aaron. <a rel="nofollow" class="external text" href="https://www.businessinsider.com/ai-model-collapse-threatens-to-break-internet-2023-8">"A disturbing AI phenomenon could completely upend the internet as we know it"</a>. Business Insider. Retrieved 2024-03-06.</cite>

</li>

<li id="cite_note-14"><a href="#cite_ref-14">^</a> <cite id="CITEREFWyllieShumailovPapernot2024" class="citation book cs1">Wyllie, Sierra; Shumailov, Ilia; Papernot, Nicolas (2024-06-05). <a rel="nofollow" class="external text" href="https://dl.acm.org/doi/10.1145/3630106.3659029">"Fairness Feedback Loops: Training on Synthetic Data Amplifies Bias"</a>. The 2024 ACM Conference on Fairness, Accountability, and Transparency. FAccT '24. New York, NY, USA: Association for Computing Machinery. pp. 2113–2147. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2403.07857">2403.07857</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F3630106.3659029">10.1145/3630106.3659029</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>979-8-4007-0450-5</bdi>.</cite>

</li>

<li id="cite_note-Alemohammad-2023-15">^ <a href="#cite_ref-Alemohammad-2023_15-0">a</a> <a href="#cite_ref-Alemohammad-2023_15-1">b</a> <a href="#cite_ref-Alemohammad-2023_15-2">c</a> <cite id="CITEREFAlemohammadCasco-RodriguezLuziHumayun2023" class="citation arxiv cs1">Alemohammad, Sina; Casco-Rodriguez, Josue; Luzi, Lorenzo; Humayun, Ahmed Imtiaz; Babaei, Hossein; LeJeune, Daniel; Siahkoohi, Ali; Baraniuk, Richard G. (July 4, 2023). "Self-Consuming Generative Models Go MAD". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2307.01850">2307.01850</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.LG">cs.LG</a>].</cite>

</li>

<li id="cite_note-16"><a href="#cite_ref-16">^</a> <cite class="citation conference cs1"><a rel="nofollow" class="external text" href="https://openreview.net/forum?id=ShjMHfmPs0">Self-Consuming Generative Models Go MAD</a>. The Twelfth International Conference on Learning Representations.</cite>

</li>

<li id="cite_note-17"><a href="#cite_ref-17">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.theregister.com/2024/01/26/what_is_model_collapse/">"What is Model Collapse and how to avoid it"</a>. The Register. Retrieved 11 July 2024.</cite>

</li>

<li id="cite_note-18"><a href="#cite_ref-18">^</a> <cite id="CITEREFGerstgrasserSchaefferDeyRafailov2024" class="citation arxiv cs1">Gerstgrasser, Matthias; Schaeffer, Rylan; Dey, Apratim; Rafailov, Rafael; Sleight, Henry; Hughes, John; Korbak, Tomasz; Agrawal, Rajashree; Pai, Dhruv; Gromov, Andrey; Roberts, Daniel A.; Yang, Diyi; Donoho, David L.; Koyejo, Sanmi (2024-04-01). "Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2404.01413">2404.01413</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.LG">cs.LG</a>].</cite>

</li>

<li id="cite_note-19"><a href="#cite_ref-19">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.theregister.com/2024/05/09/ai_model_collapse/">"Big brains divided over training AI with more AI: Is model collapse inevitable?"</a>. The Register. Retrieved 11 July 2024.</cite>

</li>

<li id="cite_note-20"><a href="#cite_ref-20">^</a> <cite id="CITEREFKirchenbauerGeipingWenKatz2023" class="citation journal cs1">Kirchenbauer, John; Geiping, Jonas; Wen, Yuxin; Katz, Jonathan; Miers, Ian; Goldstein, Tom (2023-07-03). <a rel="nofollow" class="external text" href="https://proceedings.mlr.press/v202/kirchenbauer23a.html">"A Watermark for Large Language Models"</a>. Proceedings of the 40th International Conference on Machine Learning. PMLR: 17061–17084.</cite>

</li>

<li id="cite_note-21"><a href="#cite_ref-21">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://scottaaronson.blog/?p=6823">"My AI Safety Lecture for UT Effective Altruism"</a>. Shtetl-Optimized. 2022-11-29. Retrieved 2024-06-22.</cite>

</li>

<li id="cite_note-22"><a href="#cite_ref-22">^</a> <cite id="CITEREFBorji2024" class="citation arxiv cs1">Borji, Ali (2024-10-16). "A Note on Shumailov et al. (2024): "AI Models Collapse When Trained on Recursively Generated Data"". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2410.12954">2410.12954</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.LG">cs.LG</a>].</cite>

</li>

<li id="cite_note-23"><a href="#cite_ref-23">^</a> <cite id="CITEREFDohmatobFengKempe2024" class="citation arxiv cs1">Dohmatob, Elvis; Feng, Yunzhen; Kempe, Julia (2024-02-12). "Model Collapse Demystified: The Case of Regression". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2402.07712">2402.07712</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.LG">cs.LG</a>].</cite>

</li>

<li id="cite_note-24"><a href="#cite_ref-24">^</a> <cite id="CITEREFDohmatobFengYangCharton2024" class="citation arxiv cs1">Dohmatob, Elvis; Feng, Yunzhen; Yang, Pu; Charton, Francois; Kempe, Julia (2024-02-10). "A Tale of Tails: Model Collapse as a Change of Scaling Laws". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2402.07043">2402.07043</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.LG">cs.LG</a>].</cite>

</li>

<li id="cite_note-25"><a href="#cite_ref-25">^</a> <cite id="CITEREFSeddikChenHayouYoussef2024" class="citation arxiv cs1">Seddik, Mohamed El Amine; Chen, Suei-Wen; Hayou, Soufiane; Youssef, Pierre; Debbah, Merouane (2024-04-07). "How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2404.05090">2404.05090</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.LG">cs.LG</a>].</cite>

</li>

<li id="cite_note-26"><a href="#cite_ref-26">^</a> <cite id="CITEREFGuoShangVazirgiannisClavel2024" class="citation arxiv cs1">Guo, Yanzhu; Shang, Guokan; Vazirgiannis, Michalis; Clavel, Chloé (2024-04-16). "The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2311.09807">2311.09807</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.CL">cs.CL</a>].</cite>

</li>

</ol></div></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-06-15" href="https://en.wikipedia.org/wiki/?title=Model_collapse&oldid=1295800934">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>